我们介绍了一种新型的格式转换加密,其中密文的格式隐含在机器学习的生成模型中。在这个原始的周围,我们构建了一个用于大型公共互联网平台(例如Twitter)上的秘密消息传递的系统。宽松地,我们的系统构成了经过身份验证的加密方案,一种方法是将随机密文钻头编码为生成模型的种子索引令牌分布的样品中的样品。通过修复部署方案,我们被迫考虑系统级和算法解决方案,以应对真正的挑战 - 例如接收者端解析的歧义,以及实际的代币发行的低信息携带能力〜-先前的工作。我们将GPT-2用作生成模型,以便我们的系统加密将明文Bitsring转换为适合发布公共平台的自然语言封面。我们考虑了对互联网平台内容的全面视图的对手,其目标是表面使用我们的系统进行秘密消息传递的帖子。我们进行了一套实验,以提供安全性证据,并探索运营效率和可检测性之间的权衡。
translated by 谷歌翻译
电子健康记录(EHR)系统以高频提供批判性,丰富和有价值的信息。EHR数据中最激动人心的应用之一正在开发具有来自生存分析的工具的实时死亡率警告系统。然而,最近使用的大多数生存分析方法基于使用静态协变量的(半)参数模型。这些模型不会利用时变EHR数据传达的信息。在这项工作中,我们展示了一种高度可扩展的生存分析方法,Boxhed 2.0基于模拟IV数据集的实时ICU死亡警告指示。重要的是,Boxhed可以以完全非参数的方式结合时间依赖的协变量,并通过理论来支持。我们的ICU死亡率模型实现了0.41和AUC-ROC的AUC-PRC为0.83的样品,展示了实时监测的好处。
translated by 谷歌翻译
Designing experiments often requires balancing between learning about the true treatment effects and earning from allocating more samples to the superior treatment. While optimal algorithms for the Multi-Armed Bandit Problem (MABP) provide allocation policies that optimally balance learning and earning, they tend to be computationally expensive. The Gittins Index (GI) is a solution to the MABP that can simultaneously attain optimality and computationally efficiency goals, and it has been recently used in experiments with Bernoulli and Gaussian rewards. For the first time, we present a modification of the GI rule that can be used in experiments with exponentially-distributed rewards. We report its performance in simulated 2- armed and 3-armed experiments. Compared to traditional non-adaptive designs, our novel GI modified design shows operating characteristics comparable in learning (e.g. statistical power) but substantially better in earning (e.g. direct benefits). This illustrates the potential that designs using a GI approach to allocate participants have to improve participant benefits, increase efficiencies, and reduce experimental costs in adaptive multi-armed experiments with exponential rewards.
translated by 谷歌翻译
Language Models appear to perform poorly on quantification. We ask how badly. 'Few'-type quantifiers, as in 'few children like vegetables' might pose a particular challenge for Language Models, since the sentence components without the quantifier are likely to co-occur, and because 'few'-type quantifiers are rare. We present 960 sentences stimuli from two human neurolinguistic experiments to 22 autoregressive transformer models of differing sizes. Not only do the models perform poorly on 'few'-type quantifiers, but overall the larger the model, the worse its performance. We interpret this inverse scaling as suggesting that larger models increasingly reflect online rather than offline human processing, and argue that decreasing performance of larger models may challenge uses of Language Models as the basis for Natural Language Systems.
translated by 谷歌翻译
Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, with more predictable words being processed more easily. However, evidence also shows that humans display a similar processing advantage for highly anomalous words when these words are semantically related to the preceding context or to the most probable continuation. Using stimuli from 3 psycholinguistic experiments, we find that this is also almost always also the case for 8 contemporary transformer language models (BERT, ALBERT, RoBERTa, XLM-R, GPT-2, GPT-Neo, GPT-J, and XGLM). We then discuss the implications of this phenomenon for our understanding of both human language comprehension and the predictions made by language models.
translated by 谷歌翻译
Gauge Theory plays a crucial role in many areas in science, including high energy physics, condensed matter physics and quantum information science. In quantum simulations of lattice gauge theory, an important step is to construct a wave function that obeys gauge symmetry. In this paper, we have developed gauge equivariant neural network wave function techniques for simulating continuous-variable quantum lattice gauge theories in the Hamiltonian formulation. We have applied the gauge equivariant neural network approach to find the ground state of 2+1-dimensional lattice gauge theory with U(1) gauge group using variational Monte Carlo. We have benchmarked our approach against the state-of-the-art complex Gaussian wave functions, demonstrating improved performance in the strong coupling regime and comparable results in the weak coupling regime.
translated by 谷歌翻译
我们研究了Levin(1993)所述的动词交替类的程度和句子级预测任务。我们遵循并扩展了Kann等人的实验。(2019年),旨在探测静态嵌入是否编码动词的框架选择性。在单词和句子级别上,我们发现来自PLM的上下文嵌入不仅超过了非上下文嵌入,而且在大多数交替类中的任务上达到了惊人的高精度。此外,我们发现证据表明,PLM的中间层平均比所有探测任务中的较低层都能取得更好的性能。
translated by 谷歌翻译
某些语言允许在某些情况下省略参数。然而,人类语言理解者可靠地推断出这些零代词的预期参考人,部分原因是他们建立了对哪些参考人更有可能的期望。我们询问神经语言模型是否也提取了相同的期望。我们测试了12种当代语言模型是否显示出反映人类行为的期望,这些句子暴露于Carminati(2005)中意大利五个行为实验中的零代词中。我们发现三个模型-XGLM 2.9B,4.5B和7.5B-从所有实验中捕获人类行为,而其他实验则成功地对某些结果进行了建模。该结果表明,人类对核心的期望可以从接触语言中得出,并且还指示了语言模型的特征,使他们能够更好地反映人类的行为。
translated by 谷歌翻译
强化学习中的固有问题是应对不确定要采取的行动(或状态价值)的政策。模型不确定性,更正式地称为认知不确定性,是指超出采样噪声的模型的预期预测误差。在本文中,我们提出了Q值函数中认知不确定性估计的度量,我们将其称为路线上的认知不确定性。我们进一步开发了一种计算其近似上限的方法,我们称之为f值。我们通过实验将后者应用于深Q-Networks(DQN),并表明增强学习中的不确定性估计是学习进步的有用指标。然后,我们提出了一种新的方法,通过从现有(以前学过的或硬编码)的甲骨文政策中学习不确定性的同时,旨在避免在训练过程中避免非生产性的随机操作,从而提高参与者批评算法的样本效率。我们认为这位评论家的信心指导了探索(CCGE)。我们使用我们的F-Value指标在软演奏者(SAC)上实施CCGE,我们将其应用于少数流行的健身环境,并表明它比有限的背景下的香草囊获得了更好的样本效率和全部情节奖励。
translated by 谷歌翻译
公平被广泛认为是医疗保健道德的基础。在临床决策的背景下,它取决于智力的比较忠诚(基于证据或直观),指导每个患者的管理。尽管当代机器学习的个性化力量最近引起了人们的关注,但这种认知公平是在任何决策指导的背景下,无论是传统还是创新的。然而,目前没有一般的量化框架,更不用说保证了。在这里,我们根据模型的忠诚度来制定认知公平性,这些模型是对所学的多维表述评估的,这些身份的多维表示,旨在最大程度地提高人口的捕获多样性,从而引入了代表性道德模型校准的全面框架。我们证明了该框架在来自英国生物库的大规模多模式数据上的使用来得出人口的各种表示,量化模型绩效并提出了响应良好的补救。我们提供方法作为量化和确保医疗保健认知公平的原则解决方案,并在整个研究,临床和监管领域中进行了应用。
translated by 谷歌翻译